CS-194-26: Project 3

Abhijay Bhatnagar

Part 1: Defining Correspondences

For my base morph, I'm a little shy so I will be using two random faces from the IMM database.

Face 1 and Face 2

<matplotlib.image.AxesImage at 0x138adcdd0>

For these two faces, we created a set correspondence pairs of control points using the cpselect tool.

We have 42 corresponding pairs of points set

Here we can see them plotted on their respective faces

Now we need to compute the triangulation of these points for morphing. We will use a Delaunay triangulation from scipy on the means of the pairs of corresponding points. Once we select the best set of triangles using the mean points, we will use that to index and select the triangle points w.r.t. face1 and face2.

The visualizations on face1 and face2 are provided below.

[<matplotlib.lines.Line2D at 0x138c86fd0>]

Part 2: Computing the "Mid-way Face"

Now we want to use our triangulation from the previous part to create the "Mid-way Face". First we need to define the affine function. Taking inspiration from a stack overflow post here, given a triangle defined by points (a,b,c), if we wish to transform it to a triangle defined by points (x,y,z), we will need to perform the following operation:

$$\left[ \begin{array}{ccc} t_1 & t_2 & t_3\\ t_4 & t_5 & t_6\\ 0 & 0 & 1\\ \end{array} \right] \left[ \begin{array}{ccc} a_1 & b_1 & c_1\\ a_2 & b_2 & c_2 \\ 1 & 1 & 1 \\ \end{array} \right] = \left[\begin{array}{ccc} x_1 & y_1 & z_1\\ x_2 & y_2 & z_2\\ 1 & 1 & 1 \end{array} \right] $$

or

$${\bf T A} = {\bf X}$$

which implies our transformation

$${\bf T} = {\bf X}{\bf A}^{-1} = \left[\begin{array}{ccc} x_1 & y_1 & z_1\\ x_2 & y_2 & z_2\\ 1 & 1 & 1 \end{array} \right] \left[ \begin{array}{ccc} a_1 & b_1 & c_1\\ a_2 & b_2 & c_2 \\ 1 & 1 & 1 \\ \end{array} \right]^{-1}$$

However, for the purposes of morphing, we really are interested in taking the inverse transform: $${\bf T}^{-1} = {\bf A}{\bf X}^{-1}$$

Using our affine transform, we can compute the midway between our faces. We additionally show the intermediate warp between the images and the midway point.

Part 3: The Morph Sequence

Now that we've used the affine transformation to produce a midway image, we can use a similar technique in smaller intervals to create a morphing sequence. For our two images, I'll morph 50 intervals to create the following sequence.

Part 4: The "Mean face" of a population

First, we need to load in the database of named faces. I am using the 'imm' face database, and within that database, the subpopulation of neutral men.

Loaded 33 faces and their points.

Now we need to find the average face shape, which we display simply as by its feature points.

<matplotlib.collections.PathCollection at 0x138eba390>

For more clarity, we can also show a few random faces morphed to the mean facial shape.

Original (left) and Morphed to Mean (right)

Finally, we can use all of these warped images to compute the average face of this population.

To wrap up this section, we are going to compute warps between my face and the average face of this population. I took a neutral photo to match the population, and created a matching set of corresponding points.

Using my neutral photo, I computed the warp to the mean face shape.

Original (left), Warped to Avg face (right)

As well as the average face warped to my facial shape...

Part 5: Caricatures: Extrapolating from the mean

If we take the morph of my face from the previous part, and increase the warp to beyond 1.0, we should be able to produce a caricature (that being said, it already looks rather goofy)...

Original (left), Warped to Avg face (right)

As we can see, this is quite ridiculous.

Part 5: Bells and Whistles

For my bells and whistles, I am morphing my face from neutral to a smile.